SPEC as a Performance Evaluation Measure

obtaining these measures. P otential computer system users or buyers usually employ a computer performance evaluation technique only if they believe its results provide valuable information. System Performance Evaluation Cooperative (SPEC) measures are perceived to provide such information and are therefore the ones most commonly used. SPEC measures are designed to evaluate the performance of engineering and scientific workstations, personal vector computers, and even minicomputers and superminicomputers. Along with the Transaction Processing Council (TPC) measures for database I/O performance, they have become de facto industry standards. However, do SPECS evaluation outcomes actually provide added information value? In this article, we examine these measures by considering their structure, advantages, and disadvantages. We use two criteria in our examination: Are the programs used in the SPEC suite properly blended to reflect a representative mix of different applications? And are they properly synthesized so that the aggregate measures correctly rank computers by performance? Our analysis has determined the following: